Search for: All records

Large language models (LLMs) have demonstrated an impressive ability to perform arithmetic and symbolic reasoning tasks, when provided with a few examples at test time ("few-shot prompting"). Much of this success can be attributed to prompting methods such as "chain-of-thought", which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem. While LLMs seem to be adept at this sort of step-by-step decomposition, LLMs often make logical and arithmetic mistakes in the solution part, even when the problem is decomposed correctly. In this paper, we present Program-Aided Language models (PAL): a novel approach that uses the LLM to read natural language problems and generate programs as the intermediate reasoning steps, but offloads the solution step to a runtime such as a Python interpreter. With PAL, decomposing the natural language problem into runnable steps remains the only learning task for the LLM, while solving is delegated to the interpreter. We demonstrate this synergy between a neural LLM and a symbolic interpreter across 13 mathematical, symbolic, and algorithmic reasoning tasks from BIG-Bench Hard and others. In all these natural language reasoning tasks, generating code using an LLM and reasoning using a Python interpreter leads to more accurate results than much larger models. For example, PAL using Codex achieves state-of-the-art few-shot accuracy on GSM8K, surpassing PaLM which uses chain-of-thought by absolute 15% top-1. 

Abstract Solid–liquid phase transitions are basic physical processes, but atomically resolved microscopy has yet to capture their full dynamics. A new technique is developed for controlling the melting and freezing of self‐assembled molecular structures on a graphene field‐effect transistor (FET) that allows phase‐transition behavior to be imaged using atomically resolved scanning tunneling microscopy. This is achieved by applying electric fields to 2,3,5,6‐tetrafluoro‐7,7,8,8‐tetracyanoquinodimethane‐decorated FETs to induce reversible transitions between molecular solid and liquid phases at the FET surface. Nonequilibrium melting dynamics are visualized by rapidly heating the graphene substrate with an electrical current and imaging the resulting evolution toward new 2D equilibrium states. An analytical model is developed that explains observed mixed‐state phases based on spectroscopic measurement of solid and liquid molecular energy levels. The observed nonequilibrium melting dynamics are consistent with Monte Carlo simulations.